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USING ATS-6 FOR CONTINUING MEDICAL EDUCATION 
AND HEALTH CARE IN APPALACHIA 



I 



BACKGROUND 

Many future developments in biofiedical comiunications will be based 
*on the outcomes of the demonstration programs using ATS-6# The Advanced 

Technology Satellite presents an undertaking of major significance. 

Rational and international attention is focused on this advanced 
"communication delivery system# Such heightened interest makes it all 

the more important to document and evaluate its effects on continuing 

medical educatioii and health care# 

The Appalachian region is one of three areas chosen for ATS-6 
experiments during its year of broadcasts to tlie Onited States. Its 
mountainous terrain creates remote communities^ Many of the citizens of 
these towns .receive medical care at their local Veterans Administration 
hospital* To compensate for the remoteness from large medical centers^ 
the Veterans Administration has long provided its hospital staffs with a 
variety of continuing education services. In line with this general 
philosophy, the VA wanted to test the potential for biomedical 
communication via satellite for its hospitals^ For the first time, it 
seemed possible to circumvent the problem of difficult and expensive 
communications among remote hospitals and to medical centers. The VA 
chose ten hospitals to participate in five biomedical communication 
experiments. 

Foundation for Applied Communication Technology, a non-profit 
California corporation is the VA^s administrative/coordinative agent for 
the ATS-6 project. It is responsible for the design, production, and 
distribution of weekly programs, as well as coordination of the various 
agencies, organizations, and individuals involved in the project. 
Production of films and videotapes takes place at the National Medical 
Audiovisual Center in Atlanta, Georgia. Live portions of programs 
originate from Denver, Colorado. 

Applied Communication Research, a non-profit corporation in Palo 
Alto, is the subcontractor responsible for evaluation of the VA 
sponsored ATS-6 biomedical communication experiments. Questionnaire 
"design and analysis takes place in Palo Alto. A field representative as 
well as consultants travel to the ten hospitals for interviews with the 
medical staff. An evaluation coordinator in each hospital helps 
distribute and collect questionnaires. 
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CONCEPTUAL FRAMEWORK 

Our effort to document and evaluate the effects of the ATS-6 
experioents on health care in Appalachia relies on the following 
approach: 

1« Multidimensional conceptualization of effect. 

Effect of the various biomedical communication 
experiments are explicated into four categories or 
variables* These are knowledge, attitude, utilization, 
and behavior. 

2. Multiple measurement of each effect dimension. 

Our experience in other evaluation studies warns against 
••single operationism, " that is, a ratio of one 
measurement procedure per "finding." This is necessary 
to reduce problems of acquiescence, willful distortion, 
other biases, and simple error of measurement (e.g. 
misinterpretation) . 

Since each measurement procedure embodies biases and 
defects that cannot be iletected internally, we rely on 
both direct and indirect measurement methods. 

3. "Convergent-discriminant validation" of the multiple 
measurement approaches, per finding. 

A rigorous framework for measurement validation is 
provided by the Campbell-Fiske "multitrait-multimet hod 
matrix" (1959). Like factor analysis, the multitrait- 
multimethod matrix is a preliminary analysis tool that 
distills a subset of reliable and valid measurement for- 
further analysis and reporting of findings* 

4. Use of quasi-experimental design to control both 
internal and external threats to validity while 
maintaining cost effectiveness. 

Campbell and Stanley (1963) spell out the various 
threats to validity. They enumerate eight factors 
affecting internal validity and four factors 
jeopardizing external validity. Listed below are those 
relevant for this project: 

A. HISTORY, the specific events occurring between the 
first and second measurement in addition to the 
experimental variable. For instance, a visiting 
physician may make a presentation that covers 
material also brought to the hospital via ATS-6. Or 
a nurse joining the staff might help bring about 
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certain behavioral changes* If the knowledge gain 
or new behaviors coincided with changes 'hoped for' 
in the experiments we night erroneously assume ATS-6 
programs caused the results* 

History can be called an alternative explanation* 
It is a factor influencing the internal validity of 
the experiment* And although we cannot control 
these naturally occurring events, we can monitor 
their occurrence^ We maintain an awareness of these 
events through our interviews and through hospital 
documentation such as newsletters and lists of 
continuing education events* 

B* TESTING, the effects of taking a test upon the 
scores of a second testing* Unfortunately, the 
process of measuring may change what is being 
measured* This threat is difficult to control. We 
are using a variety of measurement approaches as our 
best solution* 

C* INSTRUMENTATION, in which changes in the calibration 
of a measuring instrument or changes in the 
observers or scorers used may produce changes in the 
obtained measurements* 

We have trained our interviewer and rely primarily 
on his reports which leads to a consistency in 
question asking* Except where additional items are 
needed, we rely on a measurement instrument and do 
not change it* 

D* SELECTION, resulting from the self- initiated 
participation of certain hospitals and certain 
individuals within hospitals* This factor poses a 
threat in that participants always differ from 
non-participants* 

During our site visits, we interview both 
participants and non- participants and try to 
discover factors leading to the decision to attend 
programs, ^ 

Factors threatening external validity are: 

E* INTERACTION EFFECT OF TESTING, in which a pretest 
might increase or decrease the respondent's 
responsiveness to the experimental variable and thus 
Hake the results obtained for the pretested 
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population unrepresentative of the effects of the 
experimental variable for unpretested individuals. 

We are controlling for this threat by having only 
part of the hospital staff complete the 
pre-experiment questionnaire. We can compare their 
responses with those who complete only the 
post-experiment quest ion nai re • 

F. REACTIVE EFFECTS OF EXPERIMENTAL AR RANGEHENTS, which 
would preclude generalization about the effect of 
the experimental variable upon persons being exposed 
to it in nonexperiaental settings. 

Medical staff and patients alike are aware of their 
involvement in an experiment. Their reactions may not 
be typical of medical staff and patients in other VA 
hospitals where satellite transmissions would be 
part of a regular and continuing service. 

Since this experiment lasts for 11 months from July 
1974 through May 1975, we can assume the 
participants view it less as an experiment during 
the later, steady-state months. Data from these 
months can be compared with early data* 

G. MULTIPLE-TREATMENT INTERFERENCE, likely to occur 
whenever multiple treatments are applied to the same 
respondents, because the effects of prior treatments 
are not usually erasable. 

Many on the medical staffs of the ten participating 
hospitals see several programs. Some come to each 
week^s broadcast. From our attendance logs, we can 
send questionnaires to those representing the 
continuum from little to extensive use. 

If this were a true experiment where individuals and hospitals were 
randomly assigned to experimental or control conditions, we could easily 
eliminate threats to internal and external validity. However, the ATS-6 
biomedical communications experiment in the VA hospitals in Appalachia 
"is, at best, a quasi-experiment . Hospitals have been chosen on other 
than a random basis (see Appendix A for a list of experimental and 
control hospitals). All individuals within the hospitals have access to 
the satellite transmissions. Consequently, we must rely on careful 
construction of measurement instruments and the use of rigorous analysis 
procedures to help us overcome many of the threats. 
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TYPES OF EVALUATION 

The ATS-6 biomedical coBmunication axperiments in Appalachia are 
important as well as expensive. We feel that both fornative and 
sunmative evaluations are necessary to ensure a maximally calibrated set 
of programs evolve during the experiment and that the overall impact is 
assessed • 

FORMATIVE FEEDBACK. Perhaps the most important function of 
evaluation in a project of this kind is a continuous flow of 
effectiveness information from the evaluation team to the production 
•team. In order to be used, such feedback must follow quickly upon the 
first introduction of produced materials in the field. Formative 
evaluation procedures are designed to provide such feedback. The same 
data that serve as baselines or intermediate points for summative 
evaluation are of great value when tabulated quickly as a separate data 
series and reported back to the producers. It is acknowledged that such 
analyses are less rigorous and comprehensive than later analyses that 
will involve the same data. Turnaround time has greater primacy in 
formative evaluation than either rigor or comprehensiveness. 

The questionnaires and interviews are designed to provide useful 
information to the production team in modifying or altering material and 
presentation format. 

SOMHATIVH DOCUMENTATION. At the end of each experiment, a 
different kind of analysis is performed to assess outcomes. Because of 
limitations in ^:he quantity and quality of experimental control that is 
possible in a field experiment, the summative evaluation analysis must 
take advantage of powerful statistical procedures. Rather than focusing 
on individual programs, we will sum across programs for each type of 
experiment. Emphasis will be on outcomes of the various experiments and 
uses made of each. 



BIOMEDICAL COHBONICATION EXPERIMENTS 

The VA sponsored biomedical communication experiment using ATS-6 is 
unique in the large number of types of experiments being tried. Rather 
than focusing on one or two approaches^ there are five experiments. • 
These are: video seminars, grand rounds, teleconsultaticn, out-patient 
clinics, and computer assisted instruction. All present some fcrm of 
medical programming but vary in presentation format, techniques used,r 
target audience,, etc. A brief description of each follows: 

1. Video Seminars 

Each video seminar begins with a short (approximately 20 minute) 
film or videotape covering the program material. The medical faculty 
responsible for the content of the progtara is available at the 
conclusion of the pre-recorded portion to discuss the material. ATS-6 
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provides video and audio channels from the broadcast site (Denver) to 
the hospitals* The hospitals have telephone lines to transmit audio 
back to the broadcast site. Seminars are prepared for physicians^ 
nurses, and allied health professionals. 

2. Grand Rounds 

Grand rounds offers live presentation of patients and case 
•histories from the Denver studio. Hospitals ask questions over the 
telephone* The content is directed to physicians, nurses, and allied 
health professionals. 

3# Teleconsultation with Slow Scan 

During each of ten weekly programs, one of the VA hospitals sends 
case histories, x-rays and other static visual material to Denver via 
telephone lines using slow scan equipment that travels in a mobile van. 
Slow-scan was substituted for two-way video linkages, when frequency 
disputes eliminated two-way video for the ATS-6 experiments. The 
presenting physicians send additional materials over the phone lines. 

During the program period, a panel of 'specialists discuss the case 
with physicians at the originating hospital. The other nine hospitals 
are able to observe the visual material from the presenting hospital and 
the panelists* All hospitals can participate in the discussions using 
phone lines, although the experiment is meant to provide direct 
assistance to the presenting hospital and to provide continuing 
education to the other hospitals. 

^. Out-Patient Clinics 

Each out-pa tlfef^nt clinic begins with a short film or videotape 
covering the program material. The medical faculty responsible for the 
content of the program is available to discuss the material and to 
answer questions. 

The content is directed to patients, their families, and those on 
the hospital staff who work closely with these patients.* 

5, Computer Experiments 

The major purpose of the computer experiments is to test the 
reliability and usability of digital information transmitted via 
satellite. Two computer programs are being used ^- a Computer Assisted 
"Instruction simulation program for nurses on coronary care and a patient 
history and diagnosis program for psychiatric outpatients. The 
simulation program is being sent to two hospitals, the history/ 
diagnosis program to a psychiatric hospital. 
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EVALUATION OBJECTIVES. Although each experiment has its own 
evaluation obiectives, the following list suiRmarizes the dimensions 
involved : 

1. Demonstrate ability of satellite to mediate what is 
usually an interpersonal or face-to-face event. 

2. Establish that satellite-mediated event is judged to be 
useful by primary health care provider/consultant/patient. 

3. Establish that primary health care providers feel pertinent 
information/data is transmitted/received via satellite. 

Establish that primary health care providers feel they 
have genuine access to specialists/consultants in Denver 
(e.g* could asic questions and receive meaningful answers). 

5. On dimensions other than usefulness^ measure attitudes 
toward satellite-mediated event on the part of primary 
health care providers/consultants/patients, 

6. Determine level of use of satellite-mediated event. 

7. Determine effect of satellite-mediated event on 
subsequent know led ge/at tit ude/behavior. 

8. Determine effect of satellite-mediated event on quality 
of health care provided. 

Each of these objectives requires a consideration of the relevant 
dependent variables and the data collection/analysis methods. Again ^ 
without referring to individual experiments^ the following information 
for the eight objectives illustrates our approach: 

OBJECTIVE NO. 1: Demonstrate ability of satellite to mediate event. 

DEPENDENT VARIABLES: 

1# Responses to points covering comprehensiveness of 
information conveyed by presenting health care 
provider in initial presentation. 

2. Responses to points covering adequacy of technical 
production. 

3. Comments made during post-event debriefing. 

DATA COLLECTION AND ANALYSIS: 

Responses to points will be gathered via a brief questionnnaire 
immediately after the event* The sane type of information will 
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be gathered from those in the remote hospitals as well as fro» 
the consultants in Denver. Eiphasis is on technical adequacy 
of the event. Some interviews will be conducted innediately 
after the broadcast. They provide anecdotal evidence. 

Responses will be tabulated. Where appropriate the different 
perspectives will be kept separate (e.g. presenting hospital 
in teleconsultation experiment, consultant in Denver, non- 
presenting hospitals). Summary statement wi'll reflect on 
adequacy of the satellite to mediate the event. 

"OBJECTIVE N0« 2: Establish that satellite^mediated event is 

judged to be useful by primary health care 
provider/consult ant/ patient. 

Those involved in the experiment will have a number of attitudes 
toward the event* Host of these attitudes will be tested under 
Objective #5. However, the overriding dimension of judged usefulness 
needs to be measured throughout the experiment. Control variables 
(e.g*, background and characteristic caseload) figure prominently in the 
summative analysis of this objective. 

DEPENDENT VARIABLES: 

Respondents are asked to answer the following questions: 

1. Types of content/cases judged to be useful. 

2. Types of content/cases judged to be not at all useful. 

3# Ratio of #1 to #2 for the set of programs of each 
experiment. 

DATA COLLECTION AND ANALYSIS: 

A few items will be designed to clarify the "usef ul'*/"not 

useful" dimension. A post-viewing questionnaire as well 

as a retrospective questionnaire will obtain this information. 

Data on "useful" and "not useful" content/cases will be 
tabulated separately, then combined as a ratio. Independent 
variables of professional age, medical specialty, etc. will 
be introduced to show differentiation in "usef ul"/"not 
useful" judgments. 

OBJECTIVE N0» 3. Establish that primary health care providers 

felt pertinent information or data is transmitted/ 
received via satellite. 

This objective deals primarily with the teleconsultation experiment 
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and highlights the satisfaction or frustration in presenting cases to 
remote consultants. Technical problems associated with the 
communication linkage, as well as physical and possibly cultural 
distance separating the primary health care providers from the Denver 
specialists, may result in feelings of frustration. Apart from other 
attitudes, this objective concerns their felt efficacy in presenting 
cases. 

•DEPENDENT VABIABLES: 

The presenting health care provider discusses success in 
communicating significant aspects of: 

1. Patients* histories* 

2. Patients' present conditions and their antecedents, 
as presented verbally^ 

3. Patients" present conditions, using slow scan equipment. 

U. The range of care options that are realistic from a 
local perspective. 

5. The range of care options that particular patients 
can be committed to. 

DATA COLLECTION AND ANALYSIS: 

These questions are combined with others in a questionnaire 
that is completed by each presenting health care provider. 
The questionnaire calls for a summarization of feelings. 

Responses will be quantified and analyzed as quickly as 
possible, so the formative evaluation goals can be 
met as well as the summative evaluation goals.' 

OBJECTIVE NO. 4: Establish that primary health care providers 

feel they have genuine access to specialists/ 
consultants in Denver (e.g. could ask 
questions and receive meaningful answers) . 

Effective communication for the experiments depends on a perceived 
two-way flow. Physical and cultural distance factors as well as 
technical arrangements nay cause the health care providers to be 
*disa ppointed in the event. If questions cannot be easily asked, then 
negative attitudes may largely de^-ermine any future utilization of the 
service.* 
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DEPENDENT VARIABLES: 

The health care provider discusses success in: 

^. Asking questions, with ease* 

2# Receiving answers that clarif y/amplif y» 

•DATA COLLECTION AND ANALYSIS: 

These questions are coibined with others in a post-viewing 
questionnaire that is completed iaiaediately after each 
program. 

Interviews conducted in the VA hospitals will determine 
general feelings about ability to interact with consultants 
in Denver and overall satisfaction with access to these 
specialists* 

Responses will be quanitified and analyzed* Data from each 
program will be tabulated by hospital as well as across all 
hospitals. This information will be used for formative 
evaluation to help direct future development of the 
experiment. At the conclusion of each experiment, all 
data will be summed across programs to provide summative 
evaluation* 

'OBJECTIVE NO* 5: On dimensions other than usefulness, measure 

attitudes toward satellite-mediated event on 
the part of primary health care providers/ 
consultants/patients* 

Assuming that future utilization of satellite services depends on« a 
cluster of attitudes on the part of those in the hospitals, data 
gathered to meet this objective will primarily serve the summative 
evaluation function* 

DEPENDENT VARIABLES: 

Beyond usefulness and send-recei ving perceptions, germane 
attitudes include: 

1* Perceived acceptance of viewing/use as a legitimate 
activity for self* 

2* Perceived acceptance of viewing/use as a legitiniate 
activity for others* 

3* Reflection of viewing/use on competence* 

M* Reflection of viewing/use on pr ogressiveness* 
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5. Short-tera and long-ten implications of,^experi«ent 
for the vay ledicine is practiced in hospital. 

6. Perceived potential for satellite-mediated event. 

DATA COLLECTION AND ANALYSIS: 

Because the questionnaire needed to tape a range of 
attitudes is necessarily not briefs this objective 
will be met by means of personal interviews and mail 
questionnaires sent to a sample of health care providers 
in each site. 

An attitude battery will be constructed and will be 
pretested. Respondents will, be asked to focus on 
specific programs only as a wi^rt^lftf recalling format 
and contents Most responses wil'3rcall for generalizations 
across several programs. 

OBJECTIVE NO. 6: Determine level of use of satellite-mediated 

event. 

This objective calls for use of hospital collected information on 
number of people attending individuals programs per experiment as well 
as use of data on patterns of questionrasking. We can determine 
frequent and infrequent viewers according to medical specialties^ staff 
position^ field sites^ VA affiliation^ etc. 

DEPENDENT VARIABLES: 

1. Number of programs broadcast per experiment. 

2. Number of viewers per broadcastT. 

3. Pattern of questioning per program. 

DATA COLLECTION AND ANALYSIS: 

Data meeting this objective are drawn from attendance logs 
for the individual hospitals. The field staff will note 
discrepancies between attendance at the first of the session 
and numbers remaining at the end of the session. This kind 
of discount factor will be applied to the final analysis of 
utilization* Pattern of questioning will be determined from 
individual interviews, questionnaires, and staff in Denver. 

Utilisation findings will initially be taken as the number 
of people returning questionnaires. Attendance logs will 
correct those figures and final estimates will be based on 
what is learned about flow during the programs. Utilization 
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findings will be reported only in terms of nedical specialties^ 
staff positions r affiliation^ etc» 

OBJECTIVE NO* 7: Determine effect of satellite-mediated event on 

subsequent knowledge/a tt it ude/be ha vior» 

It is an impossible task to construct a knowledge test for each 
program that could be administered in a pre* program and post-program 
•version. The task is impossible because of the necessity of having 
information well in advance of the broadcasts and because regular 
pre-post testing leads to sensitization and multiple-treatment 
'interference* 

A second obstacle to direct pencil--and-paper measurement is the 
hospital staff themselves. We know they are unwilling to take tests on 
factual knowledge gained. Even if some consented , the evaluation staff 
would have to be concerned with the amount of time •test-taking* would 
take away from patient care. In general, the same points apply to 
attitudes and behaviors regarding specific program content. 

There does not seem to be a single solution to the inaccessibility 
of data to answer this objective. Instead we rely on pieces of 
information gathered from retrospective questionnaires, from interview's, 
and from critical incidents. We will furnish the health care providers 
with maximum opportunity to mention any knowledge gain, attitude change, 
and behavior change. 

Since films and videotapes are prepared similarly to other audio- 
visuals used and tested for knowledge gain in VA hospitals, we feel 
comfortable that knowledge is imparted this way. We will try to focus 
on expressions of how knowledge was used, etc. 

DEPENDENT VARIABLES: 

1. Kind of knowledge gain. 

2. Kind of attitude/behavior change. 

3. Amount of knowledge gain. 

4. Extent of attitude/behavior change. 

DATA COLLECTION AND ANALYSIS: 

Questions will be included in written questionnaire, when 
possible, and the field representative will elicit 
information on knowledge, attitudes, and behavior during 
interviews. 

Pinal reporting may take the form of mini-case studies. 
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rather than numerical summaries. This is especially 
reasonable since the evaluation team is unable to do long-term 
follow-up. It may be difficult to see many changes in a short 
amount o£ time. 

OBJECTIVE NO. 8: Determine effect of satellite-iediated event on 

quality of health care provided. 

Although the teleconsultation experiment intervenes directly in the 
patient care process^ most experiments provide information that may or 
may not be related to any patients currently being cared for* In these 
'cases^ it is unreasonable to expect that average patient care will be 
significantly affected by the event* 

Therefore^ we must evaluate this objective with the more subtle 
data of health care providers' perceptions and recalling of critical 
incidents. 

DEPENDENT VARIABLES: 

1. Health care providers" responses to questions about 
any perceived changes in their medical practice that 
can be traced to information acquired daring experiment. 

2* Number of critical incidents. 

DATA COLLECTION AND ANALYSIS; 

A member of the evaluation team will make appointments with 
health care providers who have participated in various programs. 
Medical faculty in Denver will be queried about possible 
critical incidents that have come to their attention. 

Data pertaining to perceived influence of event on patient 
care will be tabulated^ when possible^ using number of 
programs seen as a control variablei* Data gathered about 
critical incidents will be used as case study information. 

SELECTED PRELIHINAHY RESULTS 

We have some results from all experiments with the exception of the 
computer experiment that is just now getting underway. To date, we 
have gathered information in the following ways: 

1. Pre^experimental questionnaire directed to medical staff 
that determined demographics, major information need areas, 
important information sources, frequently used information 
sources, and attitudes toward potential of satellite-* 
mediated events. 



2. Hospital profiles of experimental and control hospitals 
with staff providing inforiaation on hospital variables 
(ager number of beds) , staff variables (number in different 
positions), patient variables (average patient load, number, 
average length of stay) , additional medical support variables 
(number of nearby hospitals, associated physicians, number 
of consultants, etc.), continuing education opportunities 
variables (library, dial access, audiovisuals , lectures) • 

3» Brief program specific questionnaires completed at the end 
of each program that determine perception of broadcast 
(technical quality, content, adequacy of discussion 
arrangements), evaluation of printed materials, and usefulness 
of information. 

^. Two retrospective questionnaires, each covering approximately 
ten weeks of programs. These determine which programs were 
seen, which were discussed with colleagues, how individual 
programs were rated, use of information, any changes brought 
about by specific programs, and programs designated as best 
or worst. 

5. Interviews with hospital staff in both experimental and 
control hospitals in which reactions, problems, and changes 
are elicited* 

6. Observations of broadcasts from which reactions, technical 
problems, and audience turnover are determined. 

Following is a sample of data from each of the four experiments 
that are currently being conducted. The complete list of topics 
presented in the broadcasts can be found in Appendix 
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VIDEO SEMINARS. Coronary Arteriography was the subject of a video 
seainar broadcast to all ten experimental hospitals on October 9, 1974, It 
vas chosen as an exaaple of a prograu that was attended by aluost equal 

"numbers of VA physicians and irurses. The total number completing the 
questionnaire at the end of the program was 200. This number is used as 
the most conservative estimate of the number of viewers. Actual audience 

'was as much as 25 percent larger. Viewers included VA and non-VA 
personnel in several staff categories. Data is presented only for the 
VA physicians and nurses. Sample size for the non-VA staff for. a 
single program was small enough to make percentages unstable* Percentages 
are shown for the most positive responses only. Total number answering 
the question is shown in parentheses below the percentage. 





H. D. 


Nurse 


Technical Aspects were GOOD 


86% 


81% 


(Total nuaber responding) 


(64) 


(68) 


Content was VBBT INTERESTING 


52% 


61% 




(63) 


(67) 


Content was VERY USEFUL 


28% 


44% 




(25) 


(27) 


Discussion Arrangements were ADEQUATE 


95% 


98% 


(6 3) 


(64) 


Printed Materials were VERY INTERESTING 


47X 


54% 




(45) 


('♦6) 


Printed Materials were VERY USEFUL 


37% 


39% 




(27) 


(23) 


Information is useful DAILY/WEEKLY 


54% 


35% 




(59) 


(62) 
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Data from a video seminar broadcast recently (March 5, 1975) is 
included for a comparison over tiie# The total audience for this 
program on Fiberoptic Endoscopy and Sigmoidoscopy wa^ smaller (120) but 
tfas also attended by nearly equal number of physicians and nurses. We 
have seen audience size decline over the months of the experiment. 
Interview data tells us the early large numbers represented curiosity 
about the satellite experiment. Audiences are now determined by 
interest in the topic and available time. This program, like the video 
•seminar on Coronary Arteriography, was designed primarily for 
physicians. 





tt. D. 


Nurse 


Technical Aspects were GOOD 


98% 


95% 


(Total number reponding) 


f41) 


(56) 


Content was VERT INTERESTING 


86X 


80% 




(37) 


(56) 


Content was VERY USEFUL 


61X 


4BX 


< — . ^ 


(18) 


(23) 


Discussion Arrangements were ADEQUATE 




100% 


(39) 


(51) 


Printed Mater^ials were VERY INTERESTING 


62% 


59% 




(26) 


(37) 


Printed Materials were VERY USEFUL 


H2% 


65% 




(12) 


(20) 


Information is useful DAILY/WEEKLY 


32X 


23% 




(3U) 


(48) 



In looking at the responses to these two video seminars^ we see 
several preliminary patterns. Until we have compiled the data series 
for all video seminars^ our remarks hold only for the programs cited. 
For the Coronary Arteriography seminar the nurses seemed slightly more 
favorable to all aspects of the program. However^ they were less likely 
to feel the information would be used on a daily/weekly basis than were 
physicians (35% versus 54%) . This is probably explainable since the 
content emphasized the physicians role. However^ it will take 
additional data to fully account for this since the nurses were more 
positive about the usefulness of the content (U4% versus 28%) and 
slightly more positive about the usefulness of the printed materials 
(39% versus 37%) k 

For the video seminar on Fiberoptic Endoscopy and Sigmoidoscopy 
the physicians seemed somewhat more positive toward the broadcast. 
Again the nurses were less likely to feel the information would be used 
on a daily/weekly basis than were physicians (23% versus 32%). This 
finding again exists along side the response from nurses that they 
found the printed materials more useful than physicians (65% versus 42%). 
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In conparing the two programs^ we see the second progra* was 
evaluated iiore highly on all dimensions except frequency of use of the 
inforaation* Individual Interviews will help us interpret these results 
that aay in fact only show rando* fluctuation around a generally high 
level of liking and use of the satellite programs* Summary data from all 
video seminars will show the strong patterns* 

GRAND ROUNDS* Surgical Treatment of Peptic Ulcers was the subject 
•of a grand rounds broadcast on September 18^ 1974* There were 209 in 
attendance* Their responses are: 





M* D* 


Nurse 


Broadcast Reception was GOOD 


69% 


82% 


(Total number responding) 


(56) 


(76) 


Broadcast was VERY WELL PRODUCED 


72% 


85% 




(58) 


(75) 


Content was VERY INTERESTING 


67% 


69% 




(57) 


(74) 


Content was VERY AUTHORITATIVE 


59% 


82% 




(54) 


(74) 


Content was VERY USEFUL 


56% 


34% 




(54) 


(73) 


Content was WORTH SEEING AGAIN 


50% 


71% 




(58) 


(65) 


Discussion opportunities were AMPLE 


72% 


90% 




(54) 


(69) 


Questions were UNDERSTOOD 


84% 


79% 




(50) 


(66) 


Questions were WELL ANSWERED 


100% 


100% 




(52) 


(67) 


Information is useful DAILY/WEEKLY 


65% 


47% 




(55) 


(66) 



The data from Surgical Treatment of Peptic Ulcers^ a greijid roundjs 
program^ shows overall satisfaction with the various aspects of the ^ 
broadcast. With the exception of usefulness of content, havings 
questions understood, and using the information on a daily/weekly basis, 
the nurses were more positive* It will be interesting wh6n we can look 
across a large number of grand rounds and determine the extent to which 
the information presented is used less by the nurses or the physicians* 
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He expect to find a matching of positive responses with the intended 
tarqet audience. For instance^ the two programs on Behavior 
Modification were intended primarily for nurses and were attended 
priaarily by nurses. In looking at the responses of using the 
inforiation on a daily/weekly basis, we have the nurses answering 
positively 49% (Behavior Modification I) and 58% (Behavior Modification 
II) while the physicians answer positively 15% (Behavior Modification I) 
and 25% (Behavior Modification II) . 

TELECONSULTATION. Technical Aspects of Dialysis was the subject 
of a teleconsultation using slow scan that was presented by one VA 
•hospital on December 18, 1974. Including VA and non-VA staff, there 
were 129 viewing the program. Following are the responses of the 
presenting hospital, the non-presenting hospitals (9) , and the 
consultants (2) ^ 





Presenting 


Hon- 


Present- 


Consul- 




Hospital 


ing 


Hospital 


tant 




n.D. 


Nurse 


n.D. 


Nurse 


K.D. 


Case presentation vas GOOD 


100X 


100% 


8 5% 


87% 


100% 


(Total nuBber responding) 


(8) 


(3) 


(40) 


(46) 


(2) 


Videotape presentation was GOOD 


100X 


100% 


88% 


96% 


100% 




(8) 




(40) 


(47) 


(2) 


Slow scan presentation was GOOD 


100% 


40% 


66% 


41% 


50% 




(8) 


(5) 


(35) 


m) 


(2) 


Interaction was GOOD 


100% 


60% 


88% 


87% 


50% 




(8) 


(5) 


(40) 


(45) 


(2) 


Potential for teleconsultation 












is HIGH 


63% 


40% 


42% 


59% 


50% 




(8) 


(5) 


(43) 


(49) 


(2) 


Potential of teleconsultation 












for non-presenters is HIGH 


50% 


40% 


43% 


45% 


50% 




(8) 


(5) 


(42) 


(47) 


(2) 



For Technical Aspects of Dialysis we find that the presenting 
}iospital, particularly physicians at the presenting hospital^ are more 
favorable to the experience than those at the other hospitals. The 
presenting physicians even evaluate the potential of teleconsultation 
for non- presenters more highly than physicians in the other hospitals. 
Since the number of medical staff at the presenting hospital is so 
small, it is difficult to make generalizations or to even put much 
emphasis on differences in perceptions between presenters and non- 
presenters other than the overall level. 
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Since the previous response pattern is specific to one program^ it 
is difficult to sense the attitudes of the consultants^ The following 
information summarizes the consultants* responses for ten presentations: 



Denver Consultants 

Case presentation was GOOD 69X 
* (Total number responding) (29) 

Videotape presentatioiv was GOOD 63% 

(27) 

Slow scan presentation was GOOD 37X 

(27) 

Interaction was GOOD 43% 

(28) 

Potential for teleconsultation is HIGH 83% 

(29) 

Potential of teleconsultation for 
non- presenters is HIGH 78X 

(27) 



The consultants seem fairly critical of the technical aspects 
of the teleconsultations. However^ they seem to feel there is high 
potential for teleconsultation in the future. Comments on individual 
questionnaires indicate the consultants have several different 
expectations for the experience that may help clarify these findings 
for our final report. 

The teleconsultation experiment is the most direct attempt to 
intervene in the medical care process. Consequently^ critical 
incidents of effects are important to the evaluation of its effectiveness. 
Following are three mini-case studies described by R.D. Hamstra, M.D. ^ 
moderator of these programs. He writes: 

case was presented to our panel of a neoplasm that by 
biopsy was a lung cancer. The lung cancer came on the 
background of chronic smoking and pulmonary symptoms 
strongly suggesting significant emphysema or chronic 
obstructive disease. When cur panel asked whether 
pulmonary function studies had been done in the patient 
before he was referred for surgery^ the response came 
that they had not been done. The panel made a strong 
point that a pneumonectomy in this patient probably 
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would prove intolerable and incompatible with life. This 
point had not been appreciated by the doctors managing 
the patient. 

second patient cones to mind in which a chest x-ray 
was presented to the panel. The chest x-ray exhibited 
a peculiar shadow along the side of the heart in the 
right heaithorax that bad not been present on a routine 
chest x-ray two years previously. It happened to be 
that one of the panelists who reviewed this case is a 
recogniled expert in what is known as the 'liddle lobe 
syndroae.* He was quite confident from the x-rays 
submitted to us and particularly through a slow scan 
magnification of the lateral x-ray^ that this patient 
did, indeed, have middle lobe syndrome. This is a 
problem characterized by lung collapse that may cr may 
not be due to airway obstruction and frequently is 
associated with complications such as abscesses, or 
recurrent infections. The presenting doctor indicated 
that the x-ray had been interpreted as a bronchogenic 
cyst. Since a bronchogenic cyst is considered to be 
benign arid no further action or followup is required, 
the management of this patient was significantly 
modified through the panelist's observation and this 
patient will be brought under more direct and continued 
medical scrutiny. 

"Another case that comes to mind is one that was presented via 
slow scan. This patient's problem had arisen within a few days 
of the time of the teleconsultation. The problem was that of 
chronic lung disease and a new infiltrate in the right lower 
hemithorax. The patient was hypoxic, and the presenting doctor 
was asking for some help. A slow scan magnification of the 
infiltrate in the right hemithorax indicated a central area of 
clearness, strongly suggesting a localized area of air-trapping. 
This pattern was a very strong indication for bronchoscopy to 
our panelists, the feeling being that some kind of obstruction to 
the airways leading to the lower right lung field was causing the 
trapped air and contributing to the persistent infiltrate. The 
presenting doctor indicated that a request for bronchoscopy 
within his institution had been rejected by the surgeon 
consultant within the last day or two. The teleconsultation 
panel unanimously and strongly supported the idea of 
bronchoscopy and it was our impression that the weight of 
the panel was sufficient to change the minds of the surgeons 
in the referring hospital so that this diagnostic procedure 
was done without further delay. 

OOT-PATIENT CLINIC. The Diabetic Patient was presented for the 
patients, their families, and those concerned with their condition on 
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Septeaber 18, 197U to 208 viewers. Of those watching the program, 42 
were there with respect to their own illness and 172 were there with 
respect to the illness of someone else* Following are their responses: 





Own 


Someone Else's 




Illness 


Illness 


•Broadcast reception was GOOD 


66% 


8U% 


(Total nuaber responding) 


(U2) 


(172) 


•Broadcast was VERY HELL PRODUCED 


92X 


83% 




(39) 


(162) 


Content was VERY INTERESTING 


95% 


82% 




(41) 


(165) 


Content was VERY AUTHORITATIVE 


16% 


69% 




(3U) 


(156) 


content was VERY USEFUL 


85% 


72% 




(UO) 


(163) 


« 

Content WORTH SEEING AGAIN 


92% 


85% 




(37) 


(lUU) 


Discussion opportunities were AHPLE 


65% 


82% 


OH) 


(lUU) 


Questions were UNDERSTOOD 


91% 


96% 




(37) 


(150) 


Questions were WELL ANSWERED 


100% 


99% 




(36) 


(142) 


Information is useful DAILY/WEEKLY 


83% 


73% 




(41) 


(147) 


The out-patient clinic program on 


the Diabetic 


Patient was 


evaluated very positively both by the 


patients and 


by the others in 


'the audience. With one exception, the 


patients responded sore 



positively than the others. Although the spread between percentages 
is soaetiies quite small, we see evidence that the target audience 
prefers the program more than others who watch. We note that the 
patients were less likely to feel there were ample discussion 
opportunities than were those viewing in respect to someone else's 
illness (65% versus 82X) . This is probably consistent with the 
other findings for this out-^patient clinic. In other words, the 
patients were more involved in the experience and wanted additional 
time to discuss the information. 
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It is importaiit for the reader to keep in mind that the programs 
chosen for inclusion are not necessarily the best nor the worst* 
Instead, we wanted to show a range of reactions to these four experiments. 
Also, the reader needs to note that all interpretation of data is 
preliminary at this point. Additional analyses will help aatplify the 
patterns. In addition^ the data presented here will be juxtaposed with 
''interview data and inforiation from retrospective questionnaires. 



ADDITIONAL INSIGHTS 

The interviews and coainents on questionnaires tell us that some 
aspects of the experiment are going very well while others are not. 
Following is a brief synopsis of these points^ They ar€ useful in 
that they provide a context for the specific responses elicited by 
individual programs. 

1. Attendance has declined over the months. Many feel that 
the medical staff have beccme more selective in choosing 
programs to attend. 

2. Some feel the two hour programs are too long. Nurses in 
particular do not like to be away from the wards that long^ 

3. Some who have requested additional information from Denver 
have found the delay time is excessive* 

4. The early programs had their share of technical difficulties. 
Some of these are related to equipment failures. Others 
seem to be related to materials presented in the videotapes 
where some viewers had difficulties seeing charts and 

gra phs. 

5. The necessity of using telephone lines has presented some 
problems. Some indicate the system is difficult to use 
(e.g. location of phone in room^ procedures for making 
call^ etc.) and others do not want to expose their 
ignorance in front of so many colleagues. 

6. A related problem is that of difficulty in understanding 
questions asked by medical staff in other hospitals. Many 
of -the physicians are not from the United States and their 
accents present difficulties. 

7. Study guides are needed sooner. 
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8. The hospital publicizes the programs to physicians in nearby 
hospitals and to the coainunity. Changes in program schedules 
Makes this hard to do* (Of course^ flexibility in meeting 
the information need ^ of the medical staff is important* Some 
program changes evol/od because additional programs were 
added at the requisst of the medical staff.) 

These problems are probably typical of a new experiment. Some have 
^already been worked out. Others are being resolved. And a few simply 
must wait while we determine their effects on the outcomes of the 
experiment. These problems need to be balanced against the many 
■"favorable comments that have been made. Following are a few of the 
ways that the experiment is making a difference. 

1. In one hospital the cardiac arrest procedure was changed 
as a direct result of the ATS-6 program on that subject. 

2. Another hospital is setting up a critical care course with 
other hospitals.' The interest grew from one of the ATS-6 
broadcasts. 

3. Most hospitals make videotapes of the programs so that 

the medical staff can view it at times more convenient than 
the live broadcast, (This positive development causes 
two problems for our evaluation. Firsts the hospitals are 
not keeping records of the number viewing the videotapes. 
Second^ many of these videotapes are being shared with 
two of our control hospitals.) 

U. Some of the medical staff have noted a new sense of closeness 
to the other VA hospitals. 

5 Panelists who remember names of the medical staff are 

greatly appreciated and seem to make a difference in the 
attitude toward the program and its content. 



NEXT STEPS 

The experimental ^^ogra»s will continue through Nay 28^ 1975. The 
last program is an open discussion on ^Use of Satellites in Biomedical 
"Communications*" All comments will be taped for further analysis. 
By early June^ the evaluation staff will be busy creating summaries 
^across all programs for each experiment type. Data from the various 
^measurement techniques will be compared using several analysis 
procedures. Control varinbles and partitioning of results will help 
clarify patterns of kn6vrledge^ attitude^ utili7ation^ and behavior 
affected by the experiment* Our final report will be ready by the 
end of July. 
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APPENDIX A: Experimental and Control Veterans Administration Hospitals 
.HOSPITALS PARTICIPATING IN SATELIITE EXPERIMENT 
Altoonar Pennsylvania 
B€ckleyr West Virginia 
•Clarksburg^ West Virqinia 
Dublin^ Georgia 
Fayetteviller North Carolina 
Hcuntain Hoie^ Tennessee 
Oteen^ Virginia 
Salea, Virginia 
Salisbury^ North Carolina 
Hilkes-Barrer Pennsylvania 

HOSPITALS SERVING AS CONTROLS FOR SATELLITE EXPERIMENT 

Bathr New York 

Butlerr Pennsylvania 

Chillicother Ohio 

Huntington^ West Virginia 

Martinsburgr West Virginia 

Murf reesboror Tennessee 



APPENDIX B: Programs Presented During Experiment 

Acute and Chronic Renal Failure (Dialysis) 
Acute Respiratory Failure (Endot rachael Intubation) 
Acute Upper GI Bleeding 
AlcoholisB Riehabilitation 
Antibiotics Uses and Abuses 
Arrhythiias (Interpretation for CCD) 
Arrhythmias (Hanageaent of Coiaonly Occuring) 
Behavior Modification Techniques 
Biofeedback 
'Blood and Gases 

Cancer Chemotherapy — Hodgkins Disease; Cancer of the Prostate 
Cancer of the Colon (Surgical Treatment) 
^Cardiac Catheterization 
Card iac Reha bilitation 
Cardiology Consultations 
Care of the Cancer Patient 
Changing Role of the Nurse 
Cirrhosis and Ascites 

Coronary Arteriography and Cardiac Catheterization 

Coronary Care Unit 

Death and Dying 

Depression 

Diabetic Patient 

Drug Abuse Rehabilitation 

Family Therapy 

Fiberoptic Endoscopy and Signpidoscopy 

Genital-Urinary Infection 

Heart Sounds 

Hypertension 

Inhalation Therapy 

Intractable Angina 

Intr a-vascular Catheterization Technique 
Neurological Consultations 

Neorological Diagnostics (Indications for Lumbar Puncture) 
Nursing, Care of the Cancer Patient 
Oral Cancer — Detection and Treatment 
Pathology^ Histology, and Tissue Consultations 
Patient/Family Orientation 
Post-op Thoracotomy Care 
Problem Oriented Medical Record 
Psychiatric Consultations 
Psycho-Physiological Process 
'Psychotherapeutic Drugs 
Pulmonary Embolism 
^Radiology Consultations 
Respiratory Intensive Care 

Solitary Pulmonary Nodule (Bronchogenic Carcinossa) 

Schizophrenia 

Stroke Rehabilitation 

Suicidal Patient 

Surgical Treatment of Peptic Ulcers 
Ultrasonic in Cardiology (Echocardiography) 
Use of Satellites in Biomedical Communications 
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